Involving stakeholders can greatly impact evaluation results. The Delphi method, a consensus-building tool, is a promising process to promote and encourage involvement from all stakeholders during the evaluation framing process. The Delphi method removes geographic challenges and time boundaries allowing all stakeholders to participate. The Delphi method uses a series of surveysinterspersed with controlled feedback designed to gather information and build consensus without requiring face-to-face meetings. Two different formats of the Delphi method, a paper-and-pencil, postal-mail version and a web-based, real-time computer version are compared in this study. Both versions of the Delphi were administered to a non-profit community based organization as part of framing an evaluation. Participation rates were better with the paper–pencil version. The quantity and quality of data collected were comparable in both versions.